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ABSTRACT 

A new spectral-spatial method for classification of hyperspec- 
tral images is proposed. The HSegClas method is based on 
the integration of probabilistic classification and shape anal- 
ysis within the hierarchical step-wise optimization algorithm. 
First, probabilistic support vector machines classification is 
applied. Then, at each iteration two neighboring regions with 
the smallest Dissimilarity Criterion (DC) are merged, and 
classification probabilities are recomputed. The important 
contribution of this work consists in estimating a DC between 
regions as a function of statistical, classification and geomet- 
rical (area and rectangularity) features. Experimental results 
are presented on a 102-band ROSIS image of the Center of 
Pavia, Italy. The developed approach yields more accurate 
classification results when compared to previously proposed 
methods. 

Index Terms — Classification, segmentation, hyperspec- 
tral images, geometrical features, rectangularity. 

1. INTRODUCTION 

The recent advances in hyperspectral remote sensor technol- 
ogy makes it possible to acquire data with a very high spectral 
(hundreds of spectral channels) and spatial (order of a meter) 
resolution. The rich spectral information of the hyperspec- 
tral data allow to discriminate different physical substances, 
leading to the potential of a more accurate classification [1]. 
However, when classifying hyperspectral images, two major 
challenges must be addressed: 1) data processing in a high- 
dimensional spectral space [2]; 2) extraction and analysis of 
spatial information [3, 4]. 

In previous works, we have proposed to apply a segmen- 
tation procedure to a hyperspectral image in order to use the 
identified regions as pixels’ spatial neighborhoods [4, 5]. Re- 
cently, we developed a spectral-spatial classification method, 
which concurrently performs best merge region growing and 
classification, yielding accurate classification maps [5]. In 
this method, the decision to merge regions at each iteration 
was based on mean spectral vectors, sizes and classification 
probabilities of the corresponding regions. 


In this work, we aim to further improve hierarchical 
model for hyperspectral image classification. For this pur- 
pose, we propose to include shape features in the region 
model. Analysis of shape features for object classification 
in remotely sensed images has been previously investigated 
in [6, 7, 8]. In all these works, segmentation was first per- 
formed, and then shape features of the detected regions, such 
as elongation, rectangularity, compactness, were analyzed. 
We propose to analyze shape features during the construction 
of a segmentation map, thus seeking for more accurate seg- 
mentation results. Thus, a new method for improved spectral- 
spatial classification of hyperspectral data is proposed in this 
paper. For a given hyperspectral image, probabilistic Sup- 
port Vector Machines (SVM) classification is first applied. 
Then, best merge region growing is performed, by iteratively 
merging neighboring regions with the smallest Dissimilarity 
Criterion (DC) and recomputing classification probabilities. 
The important novelty consists in adding shape analysis for 
computing DCs. In this paper, we include rectangularity 
feature for driving the segmentation process. 

The paper is organized as follows. In the next section, 
we present a new Hierarchical Segmentation with integrated 
Classification (HSegClas) approach. Experimental results are 
described in Section 3. Finally, conclusions are drawn in Sec- 
tion 4. 

2. PROPOSED CLASSIFICATION METHOD 

On the input a 5-band hyperspectral image is given, which 
can be considered as a set of n pixel vectors X = {xj G 
= 1,2,..., n}. The aim is to compute a classification 
map L = {Lj,j = 1,2, ...,n}, where each pixel Xj is as- 
signed to one of K thematic classes, i.e., has a class label Lj. 
The proposed HSegClas method consists of two main steps: 

2.1. Probabilistic pixelwise classification 

First, a pixelwise classification of the input hyperspec- 
tral image is performed, resulting in a classification map 
L = {Lj,j = 1,2, ...,n}, where each pixel has a unique 
class label, and class probabilities for each pixel {P(Lj = 


978-1 -4673- 1 1 59-5/1 2/$3 1 .00 ©2012 IEEE 


1409 


IGARSS 2012 



Fig. 1. Flowchart of computing a DC between two regions 
Ri and Rj. “RectSet” means a set of labels for classes which 
represent objects with high rectangularity values. 

k\xj),k = 1 , ...,AT},j = 1,2, We propose to use an 
SVM classifier, which is well suited for classifying high- 
dimensional data [1]. We refer the reader to [4] for details on 
how class probabilities are estimated using pairwise coupling 
of binary probability estimates. 

2.2. Hierarchical step-wise optimization 

At the next step, Hierarchical Step-Wise Optimization (HSWO) 
segmentation with integrated classification is performed us- 
ing the following procedure: 

1) Assign a new region label for each pixel. Each 
new region Ri gets a preliminary class label L{R{) and 
a AT-dimensional vector of class probabilities {Pk(Ri) = 
P(L(Ri) = k\Ri),k = l,...,K}. 

2) Calculate the dissimilarity criterion DC(Ri, Rj) be- 
tween all pairs of spatially adjacent regions {Ri,Rj}, using 
an eight-connectivity neighborhood (see Fig. 1): 

• First, the spectral dissimilarity DC spec (Ri, Rj) be- 
tween two regions is estimated by computing Spectral 


Angle Mapper (SAM) between the region mean vectors 

Ui = (un,...,u iB ) T andUj = (u jl ,...,u jB ) T [9]: 


DC spec (Ri,Rj) = SAM(ui,Uj ) 

Z^b= 1 u ib u jb 


arccos 


. Ef=i w |] 1/2 Ef=i u %} 1/2 


(i) 


If the regions have equal class labels L(Ri) = L ( Rj ) = 
k', 


DC(Ri, Rj) = 

(2 - max(P kl (R i ),P k/ (R j )))DC spec {u i ,u j ). (2) 

• If L(Ri) L(Rj), analyze region size: if a number of 
pixels in each region is larger than M, DC(Ri, Rj) = 
oo (the upper maximum value of float), otherwise: 


DC(Ri, Rj) = 

(2 - mm(P L{R .jR i ),P L{Ri) (R j )))DC spec (u i ,u j ). (3) 


Furthermore, for adjacent regions with non-equal class la- 
bels, their rectangularity features are analyzed. A region rect- 
angularity rect(-) is defined as a ratio of the region area and 
the area of a minimum area rectangle of an arbitrary orienta- 
tion including the region, and has values in the range (0,1]. A 
set of classes which represent objects with high rectangularity 
values is previously selected. We denote by “ RecSet ” a set of 
labels for such classes. Then, if 


(( card(R r ) > M)lk(L(R r ) E RecSet )& 

(■ rect(Ri U Rj) > rect(R r )) = true,r E {i, j}, 

then DC(Ri, Rj) = W • DC(Ri, Rj ), (4) 

where card(R) is a number of pixels in the region R, W is a 
user set parameter, W < 1. This means that if: 

• a class label of one of the regions under consideration 
belongs to the RecSet set 

• a size of this region is larger than M pixels, 

• and the rectangularity value of this region increases af- 
ter it is merged with another region under consideration, 

the DC between these two regions is reduced by a constant 
multiple. Once the DC between all pairs of neighboring re- 
gions are computed, the proposed algorithm proceeds as fol- 
lows: 


3) Find the smallest DC value DC min. 

4) Merge all pairs of neighboring regions with DC = 
DC min- For each new region R new = Ri U Rj recompute: 

• A vector of new class probabilities as 

Pk(Ri)card(Ri) + Pk(Rj)card(Rj) 


Pk {Rnew) — 
k = 1, ..., K, card(R n 


card(R n 
,j) = card(Ri 


„) 


(5) 


■ card(Rj). 
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Table 1. Results for the Center of Pavia data set. Information Classes, Number of Labeled Samples (No of samples), Global 
and Class-Specific Accuracies in Percentage. The Highest Accuracies are Bolded in Each Category. 



No of samples 

SVM 

SVMMSF 

HSWO+MV 

HSwC 

HSegClas 

Overall accuracy 

- 

94.96 

91.31 

96.67 

96.89 

97.12 

Average accuracy 

- 

92.56 

92.64 

95.41 

96.41 

97.03 

K, 

- 

93.61 

89.11 

95.77 

96.06 

96.35 

Water 

12734 

98.12 

87.99 

97.76 

97.62 

97.62 

Trees 

2405 

90.48 

89.14 

92.80 

93.64 

93.64 

Meadows 

1788 

94.08 

93.52 

91.30 

95.39 

95.39 

Bricks 

2140 

79.86 

87.87 

87.96 

91.09 

92.89 

Bare soil 

4677 

97.12 

98.00 

97.96 

99.01 

99.01 

Asphalt 

4844 

93.52 

93.04 

99.07 

96.07 

96.05 

Bitumen 

972 

82.48 

92.57 

93.21 

95.97 

99.58 

Tile 

1112 

97.41 

100 

98.61 

98.89 

99.08 

Shadows 

2020 

99.95 

91.61 

100 

100 

100 


• Class label as 

L(R n ew) = sagmax{P k (R new )}. ( 6 ) 

k 

5) Stop if each image pixel has been involved at least once 
in the region merging procedure. Otherwise, update the DC 
values between the new regions and all regions spatially ad- 
jacent to them, and go to step 3. This convergence criterion 
assumes that the image does not contain one-pixel objects. If 
it does, the algorithm may converge when [(1 — P)n\ pixels 
get involved in region merging, where P ( 0 < P < 1 ) is a 
probability of occurrence of one-pixel regions in the consid- 
ered image. 

3. EXPERIMENTAL RESULTS AND DISCUSSION 

We applied the proposed approach to the Center of Pavia im- 
age acquired by the ROSIS (Reflective Optics System Imag- 
ing Spectrometer) sensor over the city of Pavia, Italy. The 
image is of 785 by 300 pixels, with a spatial resolution of 1.3 
m/pixel and 102 spectral channels. Nine information classes 
are considered, which are detailed in Table 1 , with the number 
of labeled samples for each class. Fig. 2 shows a three-band 
false color image and the reference data. Thirty samples for 
each class were randomly chosen from the reference data as 
training samples. The remaining samples composed the test 
set. 

A probabilistic classification of the considered image was 
applied using the multiclass one versus one SVM classifier 
with the Gaussian radial basis function kernel and parameters 
chosen by five-fold cross validation: C = 128 and 7 = 2 -5 . 
Then, the proposed hierarchical optimization was performed, 
with the empirically set parameters M m 30 and W = 0.8. 
The RecSet = {7, 8 } was composed of labels for classes “bi- 
tumen” and “tiles”, representing rectangular roofs. 

Table 1 gathers global (overall, average accuracies and 
kappa coefficient [ 10 ]) and class- specific accuracies of the 
SVM classification and the proposed HSegClas method. In 
order to compare the efficiency of the proposed approach 


with previously proposed methods, we have included re- 
sults of classification using the construction of a minimum 
spanning forest from the probabilistic SVM-derived markers 
and using the SAM DC ( SVMMSF ) [4], which is a recently 
proposed and promising method. Furthermore, we have in- 
cluded accuracies of the classification by majority voting 
within the neighborhoods defined by HSWO segmentation 
with the SAM DC (HSWO+MV method) [11]. The optimal 
hierarchical level was selected by quantitatively evaluating 
classification accuracies of the considered method at several 
hierarchical levels versus the test data and retaining the best 
results. We also included results of the HSwC approach [5], 
which is similar to the HSegClas method, but does not include 
shape analysis. Figure 2(c-e) shows some of the correspond- 
ing classification maps. 

It can be seen from Table 1 that the proposed HSegClas 
yields the highest global and most of class-specific accura- 
cies. It is important to note that when comparing the HSwC 
and HSegClas results, accuracies for both classes “bitumen” 
and “tile” are improved when applying rectangularity analy- 
sis in the new HSegClas approach, while accuracies for other 
classes are similar. This proves the interest of including shape 
features in the region model for hierarchical segmentation and 
classification. 


4. CONCLUSIONS 

Hyperspectral imagery enables accurate remote sensing 
data classification, but also presents challenges of high- 
dimensional data analysis. In this paper, we have proposed 
a new HSegClas method for spectral- spatial classification 
of hyperspectral images. The method consists in applying 
probabilistic SVM classification, followed by iterative best 
merge region growing with integrated classification. The 
main novelty lies in the definition of a DC between neigh- 
boring regions as a function of region statistical, geometrical 
(area and rectangularity) and classification features. Experi- 


1411 



| Water 

■ Trees 

□ Meadows 

■ Bricks 

■ Bare soil 
| | Asphalt 

I | Bitumen 

□ Tile 

I | Shadows 



Fig. 2. Center of Pavia image, (a) Three-band color composite, (b) Reference data, (c) SVM classification map. (d) HSwC 
classification map. (e) HSegClas classification map. 


mental results did show that the proposed method succeeded 
in taking advantage of both spatial and spectral information 
for accurate hyperspectral image classification. 
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